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Abstract — The minimum distance of expander codes over 
GF{q) is studied. A new upper bound on the minimum distance 
of expander codes is derived. The bound is shown to lie under 
the Varshamov-Gilbert (VG) bound while q > 32. Lower bounds 
on the minimum distance of some families of expander codes are 
obtained. A lower bound on the minimum distance of low-density 
parity-check (LDPC) codes with a Reed-Solomon constituent 
code over GF{q) is obtained. The bound is shown to be very 
close to the VG bound and to lie above the upper bound for 
expander codes. 

I. Introduction 

In this work, we consider a family of codes based on 
expander graphs. The idea of codes on graphs was proposed 
by Tanner in 1 1 1. Later expander graphs were used by Sipser 
and Spielman in |2| to obtain asymptotically good codes that 
can be decoded in time complexity which is linear in the 
code length. By "asymptotically good codes" we mean codes 
whose rate and relative minimum distance are both bounded 
away from zero. In (|2] both random and explicit constructions 
of expander graphs were used. The explicit constructions of 
expander graphs are called Ramanujan graphs and presented in 
El, iHJ. In ||5l the construction of expander codes where both 
encoding and decoding time complexities were linear in the 
code length was presented. Though the decoder of the Sipser- 
Spielman construction was guaranteed to correct a number of 
errors that was a positive fraction of the code length, that 
fraction was small. Improvements were given in [6|, where 
the underlying graph was bipartite, and in Q- In this work, 
the distance properties of expander codes are studied. 

There are a lot of works where lower bounds on the 
minimum distance of families of expander codes are presented 
(e.g [jSJ). In these works the method proposed by Gallager 
in |l9] Ch. 2, pp. 13-20] is applied for expander codes. 
Unfortunately we were unable to find a work where an upper 
bound was derived. In this work a new upper bound on the 
minimum distance of expander codes over GF{q) is derived. 
The bound is shown to lie under the Varshamov-Gilbert bound 
while q > 32. 

It would seem the results of this work contradict the results 
of ifTOl . where a family of codes is presented (they are also 
called expander codes) which lie close to the Singleton bound 
if q is large. Nevertheless the constructions are different and 
the seeming contradiction is the result of some terminology 
confusion. 



II. Code structure 

Let G = {Vi :V2,E) be a bipartite undirected connected 
graph with a vertex set V = Vi[jV2 (Vi H ^2 =0) and an 
edge set E. Let deg (m) = AiVu e Vi, deg{v) = A2VU e 
V2, \E\ = n, then \Vi\ ^ 61, jV^al ^ &2, where 61 ^ f^, 

I, n 

''^ - a;- 

Let ¥q be a Galois field of the power q. Let us associate 
each vertex G Vi, i = 1, . . . ,bi with a linear (Ai, i?iAi) 
code cl^"* over F^; each vertex Wj g V2, j = 1, . . . , 62 with a 
linear (A2, R2^2) code Cj^\ Hereinafter cj*^ will be referred 
as constituent codes. 

For every vertex u G V, we denote by E (u) the set of 
edges that are incident with u. We assume an ordering on E. 
For a word z = (ze)gg^(whose entries are indexed by E), we 
denote by (z)^^„-) the sub-block of z that is indexed by E (u). 

Now we are ready to give a definition of an expander code: 

Definition 1: C is an expander code if 



C = {ce I : ((c)^(„^) e ) yu, e V^) A 

((cW,)eCf Vt;, £^2)} 



C is a linear code so there is a parity-check matrix cor- 
responding to it. Let h'^' be a parity-check matrix of a 

(1) tT(2) 



H^- be a parity-check matrix of a con- 



constituent code C, 

('2) 

stituent code Cj , then a parity-check matrix H corresponding 
to code C is: 



H 



where 



7T2 [diag (H('\Hf , 
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Fig. 1. Paiity-check matrix of C 



TTi is a column permutation of diag ( H]^ , , . 



,H 



uniquely defined by a graph G and by a fixed order on E. 
Remark 1: The size of H is ((1 — Ri) + (1 — R2)) n x n. 
Now we will determine the parameters of the obtained code. 
The length of C is equal to \E\ = n, the rate of C is 

R>Ri + R2-l (2) 
The equality takes place in case of full rank of H. 

III. New upper bound 

We will use the method similar to the method from ifTTl . 
Let C" be an expander code. Its parity-check matrix is given 
by ([Til. Without loss of generality Ri < i?2- The parity-check 
matrix of C can be transformed to the form: 



H 



tt(i; 



diag (h[^\h^^ 



Let C be a code corresponding to H. Codes C and C are 
equivalent hence they have the same distance properties. Now 
we are ready to prove the theorem: 

Theorem 1: Let C be an expander code, then 



d{C) < 



bi>b'> 



b'Ai 



where k = b'RiAi - (Ri - R) n, b' e N. 

Proof: Let us consider a code C of length n = b'Ai, 
b' € N. The parity-check matrix H of the code is shown in 
Fig. [1] The code C correspond to a subcode C" of C. We just 
need to add a prefix of n — n zeros to the word c of C to 
obtain the word c" of C", i.e. 



c" = (OH). 



Hence, 



d{C) <d{C")^d[c^ 



Let us consider the code C in more detail. The height h of 
its parity-check matrix and hence the number of check symbols 
in C are upper bounded with (i?i — R) n + b' {\ — i?i) Ai, so 
the dimension k of C can be estimated as follows 

k > b'RiAi - (i?i -R)n 

For the condition /c > 1 to be satisfied the following 
condition is sufficient 

b'RiAi - - i?)n > 1 
So we have such a condition 

1 



i?iAi 



After applying the Plotkin bound to C we obtain the needed 
result 



d (C) < min 



(9-1).. 



q'' -1 
{Ri-R)i 



where b' satisfy the condition -^^^ — -bi + ^ <b'<bi. 

■ 

Remark 2: In fact we can apply the stronger bound (e.g. 
the Elias-Bassalygo bound or the MRRW bound) and obtain 
a tighter bound, but even the Plotkin bound is enough for our 
purpose. 

Now we will derive an asymptotic form of the new bound. 
Theorem 2: Let {Ci}°l^ be a sequence of expander codes 
with rates RiCi) = R and lengths n{Ci) = i x 
^1, Azflthen 



LCM (Ai 



S = lim 



< 



Proof: Let us choose 

Ri-R 



q-1 fl-R 
l + R 



b'^ 



hi 



Ri 



fin), 



where / (n) 00 while n ^ 00 and / (n) = o (n), then 

^fliAi/(n)-l (-^ _ 2) 



d{C) < 



qRiAifin) _ I 

Ri — R 



Ri 



n+{f{n) + l)A, 



After dividing on n and taking the limit we have 

^^q_l(Rl_R^ 
- q \ Rl 
Finally, from conditions Ri < R2 and Q we have 

R1 + R2 ^ l + R 



(3) 



i?i < 



< 



'by LCM(a,b) we mean least common multiplier of a and b, i.e. 

LCM (a, b) = min {m : {a\m) A (b|m)} 



and after substituting it to ^ we obtain the needed result 

q-l fl-R" 



6 < 



1 + R 



IV. Lower bounds 

In this section, we obtain the lower bounds on the minimum 
distance for three code ensembles. Let us introduce needed 
notations and prove statements common for all the ensembles. 

Let (a be an ensemble of codes of length n. By A {W) 
we denote a number of code words of weight in a code 
averaged over the ensemble, i.e. 

\s\ 



where Ai {W) is a number of code words of weight in a 
code Ci E S". 

Theorem 3: If the condition 



w=i 



A{W) <1 



(4) 



is satisfied for ff then there exist a code C E <S : d (C) > d. 

d d \s\ 

Proof: J2 ^ (W^) < 1 ^ E E (W^) < Kl' which 

W=l W=li=l 

means that the total number of code words of small weight in 
<S is less than the number of codes in S', therefore there exist 
a code C E S which does not contain the words: 

d{C)>d 



Remark 3: Note, that 



A{W) 



(5) 



where Vw = {v^^^ e F^' : ||v(^)|| = W) (||v|| is the Ham- 
ming weight of v), N (S', v) is a number of codes from S" 
containing v as a code word. 

Now consider some particular code ensembles. 

A. Ensemble (oi(Ao,6) of expander codes with a Reed- 
Solomon constituent code 

Let us consider a block-diagonal matrix 

/Ho • • • \ 
Ho ■ 



H 



b — 



V 



H, 



(6) 



/ 

/ (1 



(1 — -Ro)nxn 



where Ho is a parity-check matrix of a (Ao,-RoAo) Reed- 
Solomon code over F^, n — Aob. By ip (Hb) we denote 
the matrix obtained from Hb by an arbitrary permutation of 



columns and multiplying them by arbitrary nonzero elements 



of Wq. Then the matrix 



H 



(Hb) 

^2 (Hb) 



2(l-i?n)rixn 

constructed using two matrices as layers, is a sparse parity- 
check matrix of a code from S'l (Aq, b). 

We define an ensemble S'l (Ao,&) as follows: 

Definition 2: Elements of the ensemble S'l {Ao,b) are ob- 
tained by independent choice of permutations tt^ and nonzero 
constants Cij, i = 1, 2; j = 1, 2, . . . , n, by which parity-check 
matrices of layers are multiplied. 

Remark 4: Each code from Si (Aq, b) is an expander code 
(Ai = A2 = Aq, i?i = i?2 = ^^o)^ therefore the upper bound 
is valid for all of them. 

Remark 5: \Si (Ao,6)| = {nl{q - 

Lemma 1: A number of code words of weight in a code 
averaged over the ensemble Si (Aq, 6) 

{Ai {W)f 



A{W) 



W 



where Ai (W) is a number of code words of weight W in the 
first layer 

Proof: Consider a fixed vector v*^^^ of length n, 
||v(^)|| = W. In accordance to equation ^ we need to 
calculate iV (A, v^^^). Now we consider the ensembles of 
first {Li) and second (L2) layers separately. If we know the 
number of layers from Li containing v^'^) as a code word 
(^N v*^^))) and the number of layers from L2 containing 
v^'^) as a code word (iV (L2, v^*^))), then 



N 



{W) 



N Li,v 



(W) 



(W) 



it follows from the fact that permutations and nonzero elements 
are chosen independently. For the same reason Li = L2, hence 

2 



N 



iw) \ _ 



N(Li,v 



(W) 



(7) 



To calculate iV(Li,v(^)), we proceed as follows: fix a 
permutation tti, and perform permutations and multiplications 
by constants over elements of a vector v'^'^^ but not over 
columns of a parity-check matrix. Clearly, these problems 
are equivalent, and since nothing depends on a particular 
permutation tti, we let it be the identity permutation. 

In accordance to properties of (pi there are all possible 

vectors of weight W among {(pi^^ (^'■'^'') e^ch of 

them is repeated K times, where 

K = W\{n-W)l{q-iy'-^. 



Thus, we obtain 



iv(Li,v(^)) =Ai (W) 



K 



Ai {W)Wl{n-Wy.{q-l) 



n-W 



And finally, 

N (a, v(^)) = (Ai (W) Wl {n-W)\{q~ 1)"" 



w 



One can notice that the value N ((fi, v^*^') is the same for 
aU the vectors of weight W, hence in accordance to (|5]l we 
obtain the needed result. ■ 

In the next lemma we obtain an upper bound on A (W). 

Lemma 2: A number of code words of weight in a code 
averaged over the ensemble (?i(Ao,6) can be estimated as 
follows 

where 

1 



max \y5\ogq («) " ^ log? (So (s, Aq)) 



5 =^,h, {5) = -51og, [5) -{1-5) log, [1-5)- q-ry 
entropy function and go (s, Aq) is a generating function of 
weights of code words of constituent code. 

Proof: Note that in each layer the sets of positions 
occupied by code symbols of constituent codes are disjoint. At 
the same time, all positions are covered; hence, the generating 
function of layer G (s) looks like: 



G{s)=go^o (s,Ao), 



then 



Ai (VF)= [s^] (5o^(s,Ao) 
After using an evident estimation 

'go* (s,Ao) 



Ai (W) < min 

s>0 



we obtain 



Where S = ^, h, (S) = -Slog^ (S) - {1 - S) log, (1 - S), 
F,iS,Ao)^hg (<5) + (51og, (q-l) 



2 max |^(51og, (s) - — log, {go (s, Aq)) 



Remark 6: The generating function of weights of code 
words of a (Aq, i?oAo) Reed-Solomon code can be estimated 
as follows: 

5o(.,Ao)<i+i:(^(^^°)(,-ir'^°+v), 

where do ^ {1 - Ro) Ao + 1. 

Theorem 4: If there exist at least one positive root (with 
respect to unknown 5) of equation 



then in the ensemble <Si (Ao,&) there exist codes {Ci}^^^ 



Nib) 



lim 



= 1 I such that d (Ci) > {6i — e) n, where e 



is an arbitrary small positive number; 5i is a positive root of 
equation (O. 

Proof: We just need to prove that 

The proof is similar to the proof of Theorem 2 in llTZl . We 
omit the proof here. ■ 

B. Ensemble Si (Ao,6) of expander codes with a constituent 
code from an expurgated ensemble of random codes 

In previous section we use a Reed-Solomon code as a 
constituent code. Unfortunately the length of this code can't be 
sufficiently large ( Ao < g + 1). In this section we will choose 
a constituent code from an expurgated ensemble of random 
codes and use it as a constituent code. In this case we don't 
have any constraints on the constituent code length. 

Theorem 5: For each Ao and Ro there exist a linear 
(Ao,i?oAo) code Co with such a spectrum 

1) ^0 (0) = 1; 

2) Ao [W) < 2Ao (^^^q- l)"^ q'^"^^'^-) 

for W e [l,Ao]. 

Proof: The proof can be found in ^ Ch. 2, Th. 2.4]. ■ 

The generating function of weights of code words of Co 
can be estimated as follows: 



<7o(s,Ao) < 1 



E 



2Ao n" I {q - Ifq 



- 1 Vn-'^oCl-flo) 



All the proofs here are analogical to the proofs for 
#L (Aq, 6). We will just give the main result. 

Theorem 6: If there exist at least one positive root (with 
respect to unknown d) of equation 

F2(J,Ao)=0 (9) 

then in the ensemble ^2(Ao,&) there exist codes {Ci}fJ:'^^ 
lim i^^^p'';,)! — 1^ such that d {Ci) > {62 — e) n, where e 

is an arbitrary small positive number; 62 is a positive root of 
equation (|9]l. 



F2{5,Ao) = {h, ((5) + <51og, {q-l)) 



Fi {S, Ao) = 



(8) 



2max Slog {s) - — log {go (s, Ao)) 

s>0 \ ^ Ao 



C. Ensemble (03 (Aq, b) of non-binary LDPC codes with a 
Reed-Solomon constituent code 

Let us consider the matrix 

/ 'Pi (Hb) \ 

H 



TABLE I 
Results for g = 64 



£{l — Ro)nxn 



constructed using £ layers, the notion ip (Hb) was introduced 
while defining of £"1 (Ap, b) ensemble. The matrix is a sparse 
parity-check matrix of a code from (03 (Aq, b). 

Definition 3: Elements of the ensemble (ff^ (Ao,6) are ob- 
tained by independent choice of permutations tt^ and nonzero 
constants Cij,i — l,2,...,£;j — 1, 2,...,n, by which 
parity-check matrices of layers are multiplied. 

Remark 7: The definition is similar to the definition of 
ensemble S'l (Aq, b) but the parity-check matrices here consist 
of £ layers rather than 2 ones. 

Remark 8: The codes are not expander codes and hence 
the upper bound is not valid for them. They are given here for 
comparison with expander codes. 

All the proofs here are analogical to the proofs for 
S'l (Ao, b). We will just give the main result. 

Theorem 7: If there exist at least one positive root (with 
respect to unknown 6) of equation 



F3(J,Ao) = 



(10) 

N{b) 



then in the ensemble S^{lS.t),b) there exist codes {Ci\^^^ 

N{b) _ 



lim 1^^"^°''^^ = 1 j such that d (Ci) > {S3 — e) n, where e 

is an arbitrary small positive number; 63 is a positive root of 
equation (flOl i. 

F3 {S, Ao) = - 1) {h, (S) + Slogg {q - 1)) 

+ £max [Slog {s) - log {go (s, Aq)) 

V. Numerical results 

Results obtained for g = 64 and q = 1024 are shown in 
Tables HI and HH respectively. The result for S'l, S2 and S3 are 
maximized over Aq. Note that Aq < q+1 for Si and S3. The 
derived upper bound lies below the Varshamov-Gilbert bound 
when R e (0.25; 0.89) for q — 64. This interval is widening 
while q is growing. For q = 1024 we have such an interval 
(0.05; 0.99). 

VI. Conclusion 

A new upper bound on the minimum distance of expander 
codes is derived. The bound lies below the Varshamov-Gilbert 
bound while q > 32, hence non-binary expander codes are 
worse than the best existing non-binary codes. Lower bounds 
for two ensembles of expander codes are obtained. Both of 



R 


VG 


Upper 


<3i (H) ; Ao 


X / D\ . A 

02 (ri) ; Ao 


X 1 D^ . A 
03 \ti) ; Ao 


1/8 


0.7400 


0.7656 


0.6905; 64 


0.6876; 384 


0.7355; 16 


1/4 


0.5894 


0.5906 


0.4395; 64 


0.4454; 448 


0.5860; 12 


3/8 


0.4608 


0.4474 


0.2440; 64 


0.2545; 512 


0.4585; 24 


1/2 


0.3462 


0.3281 


0.1180; 64 


0.1285; 640 


0.3445; 28 


5/8 


0.2427 


0.2272 


0.0475; 64 


0.0556; 832 


0.2415; 40 


3/4 


0.1492 


0.1406 


0.0135; 64 


0.0187; 1024 


0.1480; 52 


7/8 


0.0665 


0.0656 


0.0010; 64 


0.0030; 448 


0.0575; 64 



TABLE II 
Results for q = 1024 



R 


VG 


Upper 


Si (B) ; Ao 


h (R) ; Ao 


<53 (R) ; Ao 


1/8 


0.8036 


0.7770 


0.6590; 224 


0.6319; 192 


0.8035; 16 


1/4 


0.6573 


0.5994 


0.3350; 248 


0.3217; 276 


0.6570; 16 


3/8 


0.5252 


0.4541 


0.1440; 320 


0.1374; 304 


0.5250; 24 


1/2 


0.4028 


0.3330 


0.0545; 332 


0.0524; 384 


0.4025; 28 


5/8 


0.2884 


0.2305 


0.0180; 352 


0.0170; 384 


0.2880; 40 


3/4 


0.1817 


0.1427 


0.0045; 224 


0.0045; 640 


0.1810; 60 


7/8 


0.0835 


0.0666 


0.0005; 128 


0.0005; 768 


0.0795; 96 



the bounds lie much below the upper bound. A lower bound 
for LDPC codes with a Reed-Solomon constituent code is 
obtained. The bound is very close to the Varshamov-Gilbert 
bound and lies above the upper bound for expander codes. 
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